A Side of Data with My Robot: Three Datasets for Mobile Manipulation in Human Environments

Consideration of dataset design, collection and distribution methodology is becoming increasingly important as robots move out of fully controlled settings, such as assembly lines, and into unstructured environments. Extensive knowledge bases and datasets will potentially offer a means of coping with the variability inherent in the real world. In this study, we introduce three new datasets related to mobile manipulation in human environments. The first set contains a large corpus of robot sensor data collected in typical office environments. Using a crowd-sourcing approach, this set has been annotated with ground-truth information outlining people in camera images. The second dataset consists of three-dimensional models for a number of graspable objects commonly encountered in households and offices. Using a simulator, we have identified on each of these objects a large number of grasp points for a parallel jaw gripper. This information has been used to attempt a large number of grasping tasks using a real robot. The third dataset contains extensive proprioceptive and ground truth information regarding the outcome of these tasks. All three datasets presented in this paper share a common framework, both in software (the Robot Operating System) and in hardware (the Personal Robot 2). This allows us to compare and contrast them from multiple points of view, including data collection tools, annotation methodology, and applications.